1. Technical Field
One or more embodiments of the present invention relates to a matching device that performs matching processing of image data including an object such as a face of a person.
2. Related Art
Conventionally there is proposed a face matching device in which a face recognition technology is utilized to perform matching of a face of a person included in a face image photographed with a digital camera.
In the face matching device, the face image including the face of the person is previously registered in a database, and an input face image obtained by the photographing is matched with the database.
At this point, in the face recognition technology, it is well known that recognition accuracy depends on a photographing condition of the face image. Examples of the photographing condition include a face orientation, a facial expression, and lighting orientation and intensity.
When the photographing condition of the input face image is not matched with the photographing condition of the registration face image, because the recognition accuracy is degraded, sometimes the matching ends in failure even if the person included in the input face image is identical to the person included in the registration face image.
Therefore, conventionally the following technologies are proposed to prevent the degradation of the recognition accuracy by devising means of using the registration data.
In Japanese Unexamined Patent Publication No. 2005-115481, plural face images obtained by performing the photographing under different photographing conditions are registered in the database along with the photographing conditions, and 1:N matching is performed to the face images in the matching.
The technology disclosed in Japanese Unexamined Patent Publication No. 2005-115481 will specifically be described with reference to FIG. 22. As illustrated in FIG. 22, the face images and the photographing conditions are previously stored in a registration face image database 1021 with respect to each of persons Mr. A, Mr. B, and Mr. C.
Photographing conditions of a “front face (0 to 15 degrees)”, a “slightly oblique face (15 to 45 degrees)”, an “oblique face (45 to 75 degrees)”, and a “side face (75 to 90 degrees)” are registered with respect to Mr. A while corresponding to face images A1101, A1102, A1103, and A1104, respectively.
The photographing conditions of the “front face (0 to 15 degrees)” and the “slightly oblique face (15 to 45 degrees)” are registered with respect to Mr. B while corresponding to face images B1101 and B1102, respectively.
The photographing conditions of the “front face (0 to 15 degrees)” and the “slightly oblique face (15 to 45 degrees)” are registered with respect to Mr. C while corresponding to face images C1101 and C1102, respectively.
During the matching, a degree of similarity is determined with respect to all the registration face images, the person is identified from the photographing condition with respect to the face image having the highest degree of similarity.
In Japanese Unexamined Patent Publication No. 2006-31387, plural images are registered in consideration of fluctuating factor such as a lighting condition, an attitude, and a facial expression with respect to an individual registration person, and the matching is performed using a representative face image produced from the images.
The technology disclosed in Japanese Unexamined Patent Publication No. 2006-31387 will specifically be described with reference to FIG. 23. As illustrated in FIG. 23, the face images and the photographing conditions are previously stored with respect to each of the persons Mr. A, Mr. B, and Mr. C, and the representative face images are produced based on the face images and the photographing conditions.
A face image A1201 of a photographing condition of a “front face”, a face image A1202 of a photographing condition of a “slightly oblique face”, and a face image A1203 of a photographing condition of an “oblique face” are registered with respect to Mr. A. A face image B1201 of the photographing condition of the “front face” and a face image B1202 of the photographing condition of the “slightly oblique face” are registered with respect to Mr. B. A face image C1202 of the photographing condition of the “slightly oblique face” and a face image C1203 of the photographing condition of the “oblique face” are registered with respect to Mr. C.
A representative image D1201 is produced with respect to Mr. A based on the face images A1201 to A1203. A representative image D1202 is produced with respect to Mr. B based on the face images B1201 and B1202, and a representative image D1203 is produced with respect to Mr. C based on the face images C1202 and C1203.
During the matching, the degree of similarity between the input face image and the representative face image is determined, and the individual registration face image that corresponds to the representative face image determined as the low degree of similarity is removed from the matching target. In Japanese Unexamined Patent Publication No. 2006-31387, representative face images are hierarchically produced as illustrated in FIG. 24. FIG. 24 illustrates the state in which, in order to perform the matching processing, the representative images are produced in three hierarchical layers of a representative image in a sex unit, a representative image in an age unit, and a representative image of a person.
For example, a representative image D1231 of Mr. A is produced from individual images A1241 and A1242. The same holds true for Mr. B to Mr. G. A teenage representative image D1221 is produced from the representative image D1231 of Mr. A and a representative image D1232 of Mr. B. A man's representative image D1211 is produced from the teenage representative image D1221, a twentysomething representative image D1222, and a thirtysomething representative image D1223.
During the matching, the person is specified by sequentially comparing the input face image to each of the hierarchical layers of the representative image in the sex unit, the representative image of the age unit, and the representative image of the person.
In Japanese Unexamined Patent Publication No. 2007-226441, a feature amount vector that is obtained from the face image photographed under a certain condition (such as the lighting, the face orientation, the facial expression, and the age) is converted into a feature amount vector under another condition. Therefore, during the matching, the photographing condition of the input face image is matched with the photographing condition of the image that becomes the matching target.
For example, as illustrated in FIG. 25, a face image A1301 of a photographing condition of a “dark face” is registered in a registration face image database 1321 with respect to Mr. A, the feature amount vector is converted by a feature amount converter 1301A to obtain a face image A1311 of a photographing condition of a “bright face” and a face image A1312 of a photographing condition of a “slightly dark face”, and the face image A1311 and the face image A1312 are stored in a post-conversion feature amount database.
In Japanese Unexamined Patent Publication No. 2005-4454, only the face image suitable for the face recognition is registered in registering the face image. For example, only a front face image is registered as a condition suitable to recognize the face image. During the face recognition, a determination whether the person included in the input face image faces front is made to recognize only the face image to which the face recognition can be performed, thereby improving the recognition accuracy.
For example, as illustrated in FIG. 26, registration face images A1401, A1402, and A1403 of the photographing condition of the “front face” are registered in a registration face image database 1421 with respect to Mr. A, Mr. B, and Mr. C, respectively. The matching of an input face image P1401 of the photographing condition of the “front face” is performed using the registration face images A1401, A1402, and A1403.
In Japanese Unexamined Patent Publication No. 2000-306095, parameters such as the lighting condition and photographing direction of the input face image are estimated, a parameter adjustment is performed to the registration face image with a target set at the parameter of the input face image, the matching face image is produced, and the matching of the matching face image is performed using the input face images.
For example, as illustrated in FIG. 27, the following registration face images are registered in a registration face image database 1521 with respect to Mr. A, Mr. B, and Mr. C.
A registration face image A1502 of the photographing condition of the “slightly oblique face” and a registration face image A1504 of the photographing condition of the “side face” are registered with respect to Mr. A.
A registration face image B1501 of the photographing condition of the “front face” is registered with respect to Mr. B. A registration face image C1501 of the photographing condition of the “front face” and a registration face image C1502 of the photographing condition of the “slightly oblique face” are registered with respect to Mr. C.
During the matching, matching images A1511, B1511, and C1511 are produced from the registration face images of the photographing conditions closest to the photographing condition of the “oblique face” of an input face image P1501 with respect to Mr. A, Mr. B, and Mr. C, respectively.
For example, the matching images A1511, B1511, and C1511 of the photographing condition of the “oblique face” are produced based on the registration face image A1502 with respect to Mr. A, the registration face image B1501 with respect to Mr. B, and the registration face image C1502 with respect to Mr. C, respectively. The matching of the input face image P1501 is performed using the produced matching images.
In Japanese Unexamined Patent Publication No. 2005-56004, the registration face image under the designated condition is prepared in each registration person, and the matching is performed using the face image that is equal to the condition of the input face image, thereby improving the recognition accuracy.
The technology disclosed in Japanese Unexamined Patent Publication No. 2005-56004 will be described with reference to FIG. 28. Nine registration face images are prepared in each person with nine photographing conditions including the photographing conditions such as the “front face”, an “upward face”, a “downward face”, a “face oriented toward left”, and a “face oriented toward right”. Registration face images A1601 to A1609, registration face images B1601 to B1609, and registration face images C1601 to C1609 are registered with respect to Mr. A, Mr. B, and Mr. C, respectively.
For example, when a input face image P1601 of the “downward face” is input during the matching, the matching processing is performed using the registration face images A1608, B1608, and C1608 of the photographing conditions that correspond to the photographing condition of the “downward face”.
However, in the conventional technologies, unfortunately the recognition accuracy is degraded when the recognition is performed using the face image photographed under the photographing condition that is not registered in the database. The specific description is made as follows.
In the technologies disclosed in Japanese Unexamined Patent Publication Nos. 2005-115481 and 2006-31387, when the recognition is performed using the face image photographed under the photographing condition that is not registered in the database, possibly a person is falsely recognized as another person in which the photographing condition is registered.
The technology disclosed in Japanese Unexamined Patent Publication No. 2005-115481 will be described in detail again with reference to FIG. 22, when the face image data obtained by photographing the side face of Mr. B is input as the input face image, because the photographing condition of the “side face” is registered as the registration face image with respect to the Mr. A, the photographing condition of the input face image is matched with the photographing condition of the registration face image of Mr. A. Therefore, possibly the person included in the input face image is falsely regarded as Mr. A.
This is because the input face image of the photographing condition of the “side face” with respect to Mr. B is determined to be more similar to the registration face image A1104 of Mr. A photographed under the same photographing condition of the “side face” than the registration face image B1103 of the photographing condition of the “slightly oblique face” with respect to Mr. B.
In Japanese Unexamined Patent Publication No. 2006-31387, contents of the representative image depend on the photographing condition of the original individual face image. In FIG. 23, the individual face image A1203 of the photographing condition of the “oblique face” exists in the registration data of Mr. A. When the face image of Mr. B of the photographing condition of the “oblique face” is input as the input face image, the matching is incorrectly performed by the influence of the individual face image A1203. Therefore, Mr. B is not recognized as the matching result, but Mr. A is falsely recognized as the matching result.
In the technology disclosed in Japanese Unexamined Patent Publication No. 2007-226441, when the feature amount vector under a certain condition is converted into the feature amount vector under another condition, an error is generated from various factors, and therefore possibly false recognition is generated.
For example, because irregularity of a face depends on the person, the way of irradiating the face (the way of producing a shadow) varies even if the face is irradiated with the light having the same brightness from the same direction. Therefore, it is necessary to consider the individual variation in converting the condition in which the face is irradiated with the light. Accordingly, it is necessary that the feature amount converters 1301A to 1301C illustrated in FIG. 25 be prepared with respect to the persons Mr. A, Mr. B, and Mr. C. However, actually it is difficult to individually remove the error caused by the individual variation.
In the technology disclosed in Japanese Unexamined Patent Publication No. 2005-4454, when the face image of the photographing condition different from that of the registration data is provided as the input face image, for example, when the face image of the photographing condition of the “oblique face” is input while the face image of the photographing condition of the “front face” is registered, unfortunately the face recognition cannot be performed.
In the technology disclosed in Japanese Unexamined Patent Publication No. 2000-306095, the error is generated when the face image is produced from any viewpoint. In order to solve the problem, the images are previously prepared from plural viewpoints. However, it is unrealistic all, because the images cannot be prepared so as to satisfy the condition.
Only the matching between the pieces of registration data whose photographing conditions are matched with each other is described in the technology disclosed in Japanese Unexamined Patent Publication No. 2005-56004, and there is no description about the matching between the registration persons whose photographing conditions are not matched with each other. Therefore, it is assumed that the photographing conditions are uniformed in the pieces of registration data of all the persons. Accordingly, it is not assumed that the matching is performed using the face image photographed under the photographing condition that is not registered in the database.